Adding thermal conductivity benchmark kSRME by bpota · Pull Request #692 · ddmms/ml-peg

bpota · 2026-07-12T22:23:24Z

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.

Summary

Adding k_SRME thermal conductivity benchmark.

Adding an adapted version of the Matbench-Discovery thermal conductivity script, with a FAST option to run a lower mesh and speed up the phono3py runtime, and reference data generation script. Please let me know where I could upload/send the computed reference data. The app seems to work except the plots, but the PR can have some other bugs too.

Linked issue

Resolves #92

Progress

Calculations
Analysis
Application
Documentation

Testing

No models have been tested within ml-peg. Please let me know what is the best way to arrange this. Prediction files should be compatible with Matbench-Discovery for convenience.

New decorators/callbacks

Not yet.

ElliottKasoar · 2026-07-13T09:12:02Z

+    metrics : dict[str, dict]
+        Metric names and values for all models.
+    """
+    write_struct_info(data_path=STRUCTURE_FILE, out_path=OUT_PATH)


Is there a reason to use this rather than get_struct_info, which also has a write info option?

This would save reading STRUCTURE_FILE an additional two times as part of get_system_names and get_system_ids.

I don't think there should be any reason against changing. Thanks for the suggestion!

ElliottKasoar · 2026-07-13T09:13:40Z

Thanks for this, @bpota! We'll try to take a detailed look as soon as we can.

Regarding the data, if you want us to upload it to the S3 bucket, you can share the data with me or @joehart2001 in whatever way is easiest (here/OneDrive/Dropbox/email/Slack) and we can upload it for you.

joehart2001 · 2026-07-14T13:10:28Z

I think a good way to do the fast only and full tests would be to:

have general running function
mark the full test as very_slow like e.g. phonons
this means when you run the test normally, on the fast one will run, and if you use the very slow cli flag then both will run

bpota · 2026-07-14T20:53:32Z

I think a good way to do the fast only and full tests would be to:

have general running function

mark the full test as very_slow like e.g. phonons

this means when you run the test normally, on the fast one will run, and if you use the very slow cli flag then both will run

Thanks @joehart2001, I agree this should be easier. I will change it accordingly.

joehart2001 · 2026-07-16T14:04:11Z

The run failed at the very end (after all 103 materials had been computed and saved) when writing the combined fast_kappas.json.gz / kappas.json.gz: ValueError: cannot insert material_id, already exists

each per-structure result dict already contains a material_id key, so pd.DataFrame(...).T has a material_id column. The code then set the index name to material_id and called reset_index(), which tries to promote that index into a second material_id column -> pandas rejects the duplicate

added code suggestions above for this

bpota · 2026-07-21T21:38:45Z

Ideas for interactivity after discussing with @joehart2001:

(Fast) kSRE: Predicted thermal conductivity against reference on a log-log scale.
(Fast) kSRME: Same violin plot as in the case of phonons.
Individual datapoints for both kSRE and kSRME: Predicted mode thermal conductivity against reference on a log-log scale.

- Add app_thermal_conductivity.py Dash sub-app - Replace unicode κ with k in docstrings/labels for encoding safety - Sort DataFrames by index before metric computation for determinism - Fix output filename: figure_conductivity -> figure_thermal_conductivity - Add OC157 dvc.yaml, update uv.lock, add CLAUDE.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S7VGFQPPbiFQsxbHa1yKbu

Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>

…ompleted ones

…benchmark

ElliottKasoar requested review from ElliottKasoar and joehart2001 July 13, 2026 09:00

ElliottKasoar added the new benchmark Proposals and suggestions for new benchmarks label Jul 13, 2026

ElliottKasoar reviewed Jul 13, 2026

View reviewed changes